Add span-creation JMH benchmarks (front-half allocation)#11915
Conversation
Adds application-thread (front-half) allocation benchmarks that ground the TagMap 2.0 / SpanPrototype work against the real span lifecycle: - SpanCreationBenchmark (front-A): single span create -> tag -> finish. Bare baseline (startSpan vs buildSpan), two known-tag shapes set after start — web-server (7 tags) and JDBC/DB client (9 tags) — plus a builder-tag-path arm (withTag before start, the OTel-bridge shape) to track how the startSpan / buildSpan lineages diverge across releases. - TraceAssemblyBenchmark (front-B): web-shape root + N children (childCount 1/5/20), exercising the per-span baseline-tag copy that map-to-map copy (TagMap 1.0) and the trace/span tag split (level-split) target. - DropWriter: shared no-op Writer so finish() excludes serialization / agent I/O, isolating front-half allocation for -prof gc. Deliberately drift-stable (v1.53->master API) so the same file can be grafted onto old release tags for a historical allocation curve. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Runs startSpan create+finish on a virtual thread — the regime the platform-thread SpanCreationBenchmark is blind to. startSpan's thread-local SpanBuilder reuse (#9537) is disabled on virtual threads, so a builder is allocated per span there; the builder bypass (#9998) removes it. This bench is where that difference shows. Starts the virtual thread via reflection (Thread.startVirtualThread) so the jmh source set still compiles on pre-21 toolchains; requires a 21+ JDK at run time. The per-op vthread spawn/join cost is constant across tracer versions, so it cancels in the version-over-version delta. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dougqh
added
comp: core
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
Fold the v1.53.0 -> 1.64.0 sweep results into the SpanCreationBenchmark and TraceAssemblyBenchmark javadoc so the trend is discoverable from the source without re-running. Allocation (gc.alloc.rate.norm B/op) carries a net Δ%; throughput (ops/us) is included but flagged directional-only and given no Δ% since it has no reliable trend (laptop thermals + per-fork inlining bimodality). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
There was a problem hiding this comment.
More details
Pure benchmark-only PR (no production code changes): all four new JMH files are measurement infrastructure. Every critical API contract was verified statically — DropWriter correctly implements all five Writer methods, all Tags constants exist, asChildOf(root) compiles because CoreTracer.buildSpan() returns the concrete CoreSpanBuilder (which has asChildOf(AgentSpan)) not the SpanBuilder interface (which has only asChildOf(AgentSpanContext)), and the Thread.startVirtualThread MethodHandle pattern was confirmed correct on JDK 21. No behavioral regressions are possible: this PR adds only benchmarks.
📊 Validated against 6 scenarios · Open Bits AI session
🤖 Datadog Autotest · Commit e92967b · What is Autotest? · Any feedback? Reach out in #autotest
1 participant
What This Does
Adds application-thread ("front-half") allocation benchmarks for the full span-creation lifecycle (create → tag → finish), to ground the TagMap 2.0 / SpanPrototype allocation work against the real span path.
SpanCreationBenchmark— single span. Bare baseline (startSpanvsbuildSpan), two known-tag shapes set after start — web-server (7 tags) and JDBC/DB client (9 tags) — plus a builder-tag-path arm (withTagbeforestart(), the OTel-bridge shape).TraceAssemblyBenchmark— web-shape root + N children (childCount1/5/20), exercising the per-span baseline-tag copy that map-to-map copy (TagMap 1.0) and the trace/span tag split (level-split) target.SpanCreationVirtualThreadBenchmark—startSpanon a virtual thread (JDK 21+ via reflection), the regime whereSpanBuilderreuse is disabled.DropWriter— shared no-opWritersofinish()excludes serialization / agent I/O, isolating front-half allocation for-prof gc.Motivation
gc.alloc.rate.norm(B/op) is deterministic and is the primary signal; throughput is directional. The benchmarks are deliberately drift-stable (only API byte-identical v1.53→master), so the same file can be grafted onto past release tags to reconstruct a historical allocation curve.Additional Notes
-prof gc, and force tracer logging to WARN (unsuppressed DEBUG logging allocates and corrupts the numbers).🤖 Generated with Claude Code
Validation — rigorous 8-thread version sweep (1.53 → 1.64)
These benchmarks were run across every release 1.53→1.64 (
-f3 -wi5 -i5 -t8 -prof gc) to validate the allocation trend. Allocation is the trustworthy axis; throughput is directional (build-to-build inlining-bimodal on a laptop — read trends, not points).Allocation (B/op) — trustworthy axis
bareStartSpanbareBuildSpanwebServerSpanwebServerSpanViaBuilderjdbcClientSpanwebServerTrace[1]webServerTrace[5]webServerTrace[20]Cumulative 1.53 → 1.64:
bareStartSpanbareBuildSpanwebServerSpanwebServerSpanViaBuilderjdbcClientSpanwebServerTrace[1]webServerTrace[5]webServerTrace[20]Two structural steps carry the curve: 1.59 (TagMap 1.0 default → Entry-sharing) and 1.61 (interceptor/links elimination cluster), plus a 1.63 jdbc-specific drop. Single-span −28-31% (triple-confirmed: this sweep, the 4-thread archaeology, and the PetClinic macro stress test at −29%). Trace path smaller (−12-20%) — the per-child fresh-tag insertion cost that TagMap 2.0 (dense + level-split) targets next.
Throughput (ops/us) — directional only
bareStartSpanbareBuildSpanwebServerSpanwebServerSpanViaBuilderjdbcClientSpanwebServerTrace[1]webServerTrace[5]webServerTrace[20]Caveat: throughput on this harness is build-to-build inlining-bimodal (see the bareStart column alternating ~4.2 / ~5.5) — reproducible across clean re-runs, not thermal or contamination, and not stabilized by
@Fork(3)(which averages within a build, not across builds). Read trends, not points. The one real-ish signal is the 1.62→1.64 climb on the tagged arms (web 3.4→4.6, webBuilder 3.5→5.3), corroborating the 1.62isOutbound/span.kind CPU win (#11116) + client-stats changes. Allocation is the axis; throughput corroborates large effects only. A CPU win should be sized from a profile (cycles), not this axis.